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ABSTRACT 


This thesis addresses the benefits that the employees of the Naval Avionics Center 
(NAC) believe they are receiving from management training. Specifically, this study 
determines whether or not the management training that is offered to NAC scientists and 
engineers is perceived as beneficial. In order to determine the perceptions of the trainees, 
a questionnaire was developed and distributed to all eligible employees. Fifty-nine percent 
of the participants responded. The results of the research indicated that the trainees 
generally had positive responses to all of the survey questions. Based on those results, it 
is concluded that the scientists and engineers do perceive the training that they are 
receiving as beneficial. However, due to the small number of surveys retumed, the results 


for the entire study must be considered exploratory. 
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I. BACKGROUND 


a NAVAL AVIONICS CENTER 

The Naval Avionics Center (NAC), which is located in 
Indianapolis Indiana, is a non-profit government organization 
that specializes in the field of aviation electronics, or 
wavrenics:.” They cater not only to the specific needs of the 
four American services--the Navy, Marines, Air Force, and 
Army--but to international customers as well. Their work 


force consists of over 3400 employees, 34% of which are 


scientists and engineers. The Center 1s organized into nine 
departments. (A basic organization chart is provided as 
Appendix A.) Four of these departments--department "200" 
(Weamueacturing Technology), -"400"- (Product inteqrity and 
Assurance), ODO (Systems and Technology), and "900" 
(Engineering) --are where a majority of the scientists and 
engineers are located. As civil servants, they are salaried 


employees who are paid on standard government GS/GM pay 
scales. 

NAC’s mission is "To conduct research, development, 
engineering, material acquisition, Diuvot and limited 
manufacturing, technical evaluation, depot maintenance, and 
integrated logistic support on assigned airborne electronics 


(avionics), missile, spaceborne, under sea and surface weapon 


systems and related equipment." Peet. 18] With a 
mission statement as diverse and technically oriented as 
NAC’s, management training becomes an important issue in the 
development of their engineers. 
1. NAC Management Training 
NAC management currently spends over ten million 
dollars annually to offer a variety of training opportunities 
for their employees. A large portion of the training budget 
is allocated to management training as can be attested to by 
the vast number of management courses available to NAC 
personnel (see Appendix B). During the past couple of years, 
NAC has been placing increasing emphasis on Deming training, 
otherwise known as Total Quality Management. The acceptance 
of this philosophy solidifies NAC’s commitment toward training 
and to the continual learning of their employees. 
Instead of NAC employees taking courses piecemeal 
and without any guidance, NAC management has developed a 
program called the "Individual Development Plan" (IDP). The 
IDP is a program that identifies, through both long and short 
term goals, the employee’s desires for training development. 
If the employee elects to take part in the educational 
opportunities offered by NAC, then the IDP becomes an annual 
process by which the employee and the supervisor sit down and 
try to determine what type of specific training courses would 


be best for the employee to take. It is up to the individual 


employee to decide which general route will be followed--i.e., 
technical or managerial. Once that decision has been made, 
the employee and the supervisor meet and try to develop a plan 
that best fits his/her individual needs. 

NAC’s Current management training program 
encompasses supervisory training development, management 
development and executive development. These broad categories 
can be broken down further to include such generic courses as 
General Management Skills and Managerial Aspects of 
Supervision, or courses that are designed to teach more 
specific skills such as interviewing, negotiating, or 
appraising performance. 

2. Corporate Vision 
NAC realizes that its sole purpose for existence is to 
support the Fleet. In order to maximize the effectiveness of 
that support NAC has identified its corporate vision: "to be 
recognized as the leader in avionics and manufacturing 
excellence." [Ref. 1] To help achieve the corporate vision, 
The following excerpt from the Center’s overview statement 
describes the importance NAC management places on the 
Gogtrinuing education and training of its scientists and 
engineers: 
the center invests in a strong personnel training 
program designed to foster technical and managerial skills 
especially attuned to addressing the Navy’s airborne 
electronics issues of today and tomorrow. In ‘order to 


stay abreast of new philosophies in the systems 
acquisition process and the rapid advances in avionics 


technologies, the Center continually invests in the 
upgrading of its personnel’s capabilities. [Ref. 1] 


With this type of philosophy toward training and with the 
training that is currently being offered, NAC is taking 


positive steps toward the realization of that vision. 


B. PURPOSE AND SCOPE 

NAC management is committed to the further development and 
improvement of their employees. This is demonstrated through 
their $10M annual training budget and their continual interest 
in providing their employees with ample Craunuenc 
OppoEtunit wes: The scientists and engineers at NAC are 
offered training to further their technical abilities as well 
as develop their managerial skills. For the purposes of this 
study, the focus is limited to management training. N A C 
offers a wide variety of management training courses to their 
scientists and engineers. This study evaluates the 
effectiveness of that training from the perspective of the 
training participants, i.e., the managers or potential 
managers. "A manager," as defined by R. W. Griffin, "is 
someone who plans and makes decisions, organizes, leads, and 
controls human, financial, physical, and information 
resources, with the aim of achieving organizational goals, in 
an efficient and effective manner." [Ref. 2] How does 
a person acquire the skills necessary to become a successful 


manager? The most common path is through a combination of 


experience and proper management training. [hes ieaining 
should not only provide for the accomplishment of managers’ 
tasks as outlined above, but it should also be relevant to 
their jobs, as well as beneficial to themselves and their 
organization. This study addresses the benefits the people 
believe they are receiving from management training. 
Specifically, this study determines whether or not the 
management training that is offered by NAC to scientists and 
engineers is perceived as beneficial. 

Recently, NAC management requested that the Naval 
Postgraduate School conduct an evaluation of the return on 
investment (ROI) of NAC training and this study represents a 
part of that overall effort. Evaluating management courses is 
erp roult -@t best. ANS a TEesult» this effort’ wealvizes 
perceptions of trainees to assess training outcomes, which can 
then be reviewed by NAC management to make judgements about 
mine payotft of the training program for the organization. 

This study does not address specific cost benefits as that 
is the purpose of another of the efforts in this program of 
research. Rather, the issue of interest is the perceived 


outcomes of specific courses. 


C. RESEARCH QUESTIONS 
The primary research objective addressed in this thesis is 
whether or not specific management courses are perceived as 


useful by NAC scientists and engineers. To accomplish the 


research objective, the following specific research questions 
about training outcomes are addressed: 


ili How relevant iS management training to job 
requirements according to trainees? 


2. How satisfied are trainees with specific management 
courses? 


3. What is the perceived relationship between management 
training received and changes in job performance? 


4. What are the potential benefits of the management 
training for the NAC organization? 
The assessments can be used in conjunction with other studies 
to provide NAC management with a basis on which to evaluate 


its return on investment for training. 


D. ORGANIZATION OF THE THESIS 

This thesis is divided into five chapters and three 
appendices. This chapter is followed by Chapter II, which is 
a review of the literature that is pertinent to the research. 
Chapter III provides the data base for this research and 
discusses the method used for the development and 
administration of the questionnaire. Chapters IV and V 
provide the results and conclusions, respectively, from the 
analysis of the questionnaire data. Chapter V also includes 
recommendations for NAC management training. The Appendices 
consist of NAC’s organizational chart, the list of management 


courses available, and the questionnaire used for this study. 


2 om LITERATURE REVIEW 


A. MANAGEMENT TRAINING 
<, tntroaucts0n 

"No resource iS more critical to the success of an 
organization than management talent. And no resource is more 
scarce. Effective managers are made, HOt Bown. 
[Ref. 3] For effective managers to be "made," some 
form of training must be implemented. William Thomas, author 
of "Training and Development Do Make Better Managers," 
[Ref. 4] says that: 


Mine success Of any kind of education “or traaning 1s 


difficult to measure. Companies that invest time and 
money in developing their managers often see improvements 
in overall Company @s Perrormance-. However, unlike 
manufactured products, good managers cannot be mass 


produced. It is the combination of personality, instinct, 
on-the-job experience, good mentoring, solid educational 


background, decision-making ability, and luck that makes 
a good manager." 


Thomas goes on to say that most successful large companies, 
companies like IBM, Walt Disney Productions, GE, and Dana 
Corporation, provide their managers with "educational 
luxuries." These luxuries include job rotation, succession 


planning, individual coaching, and, usually, sophisticated in- 


house training programs. Peters and Waterman in their book, 
"In Search of Excellence," identified 71 companies that they 
felt were the most successful in the country. Thomas 


conducted a study of those companies and found that 97% have 
their own in-house management development programs, 96% pay 
for their managers to take courses in management from local 
colleges and universities, and almost half of those companies, 
43%, grant time off for study leave. [Reprre4 7p 234 

Through his study, Thomas further concluded thawwor 
those companies offering management development courses, 94% 
provided their own training staffs. By using their own 
personnel as instructors, the company’s "corporate culture” 
can be transmitted to the managers and supervisors of the 
future. [Ref. 4, p. 53] Corporate culture is an important 
element to convey because it instills within the recipient a 
sense of pride and commitment to the organization. 

Lastly, Thomas states that "Management education is 
cost effective only if you think it is. Its payoff lies in 
the future. It is an investment in the capital resource of 
leadership, of providing educational breadth to enable a 
person to achieve managerial depth." [Ref. 4, p. 53] It is 
hoped that through the investment in training, successful 
managers are made. As stated previously though, being 
successful is the combination of personality, instinct, on- 
the-job experience, good mentoring, solid educational 
background, decision-making ability, and luck. However, if a 
continuing and sound - educational/training program is 
implemented it can go a long way toward making successful 


managers. 


2. Management Training in The Workplace 

a. In-House Training 
With jobs in today’s workforce becoming 
increasingly more complex, there is greater concern over 
whether the education and training received by the workers 
will fulfill the requirements needed for them to function 
successfully in their chosen fields. According to Donald 
heoerpatrick, an organization can improve its supervisors’ 
performance through proper training. He states that there are 
three major in-house approaches for training and developing 
supervisors: selected reading, on-the-job coaching, and 

classroom programs [{Ref. 5]. 

(1) Selected Reading 

Ths ws probably the most ditticult of. the 
three to accomplish because encouraging supervisors to learn 
through reading is not an easy task. It requires books or 
articles that are easy to read and readily accessible, but 
there must be some motivational factor built into the reading 
program. (Ref. 5, p. 52] The reading material should cover 
topics such as leadership, motivation, communication, decision 
making and problem solving, time management, and subordinate 
Ea Livy nc. To stimulate participation and to motivate 
Supervisors to read, the training/personnel department, as 
well as higher-level managers should also become involved in 


the program. This involvement can consist of (1) asking 


supervisors to read certain books, then leading them in a book 
discussion; (2) discussing what was read in the book and how 
it could be applied to the workforce; and (3) having 
Supervisors read chapters of a book and present a summary to 
each other. [Refi woe 6754 

(2) On-The-Job Coaching 

"On the job training takes place all the time: 
Managers are always training subordinates by setting examples. 
Training by example is one of the most powerful training 
techni quess. coaching reraniorcecai te [Ref. 5 pems2) 

In their every day job, managers coach 
supervisors in a number of ways: (1) by correcting mistakes, 
(2) through positive reinforcement, and (3) by completing the 
employees formal performance appraisal. To become more 
effective at appraising an employee’s performance or 
conducting proper coaching procedures, Kirkpatrick recommends 
the following steps: 

- Clarify what is expected of the employee. 
- Tell the employee how he/she is doing. 


- Select one or more performance areas in which the employee 
needs to improve. 


- Develop a performance improvement plan. 

- Coach to implement the plan. 
Managers should be schooled in the proper principles and 
techniques of coaching in order to get the most out of their 


employees. 
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(3) Classroom Programs 

If a company is large enough, classroom 
programs should be conducted in-house. The term "in-house" 
doesn’t necessarily mean that the program takes place on the 
company premises. It simply means that attendance is 
restricted to managers from one organization. The main 
advantage of in-house programs is that the course or courses 
being taught can be structured to meet the specific needs of 
the organization. Earry Jand ~Robinson o([Rei. 3, pp. 6-13) 
Geseribe classroom programs that are run by outside 
organizations as taking the Johnny Appleseed approach. la 
you scatter enough seed on enough ground, some will take root 


Lal 


and produce. Programs are offered by outside organizations 
such as colleges and universities, the American Management 
Association, and trade associations. In creating a good in- 
house training program, Kirkpatrick identified three phases 
that must be followed: planning, conducting, and evaluating. 
[Ref. 5 p.- 54] See Table 1 for an outline of the 
characteristics that help to make a good in-house training 
program. In-house programs can develop, within the employee, 
the attitudes that the organization wants. If set up 
properly, the supervisor’s knowledge will increase and his or 
her skills will improve. 


This same point is made again by Anthony P. 


Carnevale. Management training for middle managers is common 


1 a 


TABLE 1 


GOOD IN-HOUSE TRAINING PROGRAMS 


ees ae 


Based on needs Based on chijectives Convenient to learner Adequate facilities 
Stated in terms of Understandable to Proper length Minimum 

expected outcomes learner Proper frequency interruptions 
Learner-criented Pertinent te learner 

Clear to everyone Of benefit to learner 


CONDUCTING 


Knowledgeable in subject Varied Audio 

Interested in teaching Maintain learner interest Video 

Respected by learners Get group involvement Simple 

Skilled in communicating Help learners learn Minimum cost 
Skilled in teaching Help learners learn 


EVALUATING 


How well did To what extent did To what extent did What final results 
learners like the participants learn jek behavior change were achieved? 
course? facts, principles, because of the 

skills ana course? 
attitudes? 





in American companies. Although new managers usually start 
with a good educational base, they often have little training 
in motivating and managing people. Consequently, as they move 
from their areas of technical expertise and begin to take on 
the responsibilities of coordinating resources and people, 
they usually find that their managerial skills are lacking. 
To alleviate this problem, American employers try to "make" 
managers through an extensive employer-provided training 
programs, including on-the-job training “and mentoring 


{Ref. 6] 


eZ 


b. Costs/Effects of Training 

In a 1984 survey focusing on formal employer- 
sponsored training, Carnevale and Garner estimated that 
employers spent approximately $30 billion in direct costs for 
formal training during that same year. [Ref. 7] This 
is consistent with another 1984 survey in which similar 
figures were obtained. [Ref. 8] 

Barron and Black took the above study and went one 
step further. Instead of looking at training costs, they 
wanted to take a look at the effects of training on 
employment, i.e., productivity and wages. What they concluded 
was that a 10 percent increase in the amount of time devoted 
to training was associated with an average 3 percent increase 
in productivity and a 1.5 percent increase in wages. 
(eet... .9} 

After evaluating the above mentioned studies, Gary 
Becker, in his article "What We Do and Don’t Know About 
Training in the Workplace," [Ref. 10] Gone luded. OU ehat 
there are some issues concerning training that are consistent 
from one company to another. These consistencies include: 

- The likelihood of receiving training declines with age. 


- The likelihood of receiving training increases with 
education. 


- Men are more likely to receive training. 


Wavoce Eraanand 7S informal. 
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In addition to finding out what we know about training, Becker 
also found out what we do not know: 
- We have no universal definition of training. 


- It ais unclear how to measure whether training is 
successfully. 


- Changes in training over time are unknown. 
3. The Future of Management Training 
@. Increased Management Personnel 

"In the future there will be more management and 
more managers." [Ref. 6, p- 21] Anthony Carnevale believes 
this because, as seen over the past decade, a majority of the 
jobs in today’s organizations that require less skill have 
been replaced by jobs using sophisticated technology, which 
require more highly skilled employees. According to (me 
Bureau of Labor Statistics, between now and the year 2000 the 
number of managers, professionals, and marketing personnel 
will increase by more than 30 percent. The number of 
management jobs is expected to increase by three million, more 
than one and a half times the rate of increase in other 
CeCCUpatL2 onus - [Ref. 11] The question is this, will 
management training have to change in order to keep pace with 
the increasing changes in technology? 

b. Changes in Training 

David Burgoyne, author of "Management Development 

Programs: Then and Now," answers the above question in the 


negative. According to Burgoyne, “Although the change: 


14 


businesses experience in the next few decades may be massive, 
the role of management development programs will be a static 
one: to prepare managers to handle change and challenges more 
effectively." [Ref. 12] 

One of the best places for a new manager to learn 
how to perform many of the necessary management skills is on 
the job. Sia kis such as leadership, communication, 
interpersonal effectiveness, teamwork, negotiation, learning, 
counseling, and teaching are difficult to teach in school but 
through applied learning, are easy to teach on the job. This 
does not mean however, that we should leave the development of 
new managers to the accidents of experience. American 
companies are better at providing applied learning than the 
schools. The first challenge employers must face is the 
difficult task of teaching the schools a more applied approach 
to management education and to encourage the schools to add 
eurriculum in the “soft” skill areas. It will not be enough 
foe the organizations to duplicate the academic curriculums, 
they must add the applied dimension to their training to be 
effective. [Ret 6, ps428)] 

Hawkins and Barclay describe the changes in 
technology as "extremely complex, inter-disciplinary based and 
Constantly evolving, through radical and refining changes, in 
response to the needs of the business." [Ref. 13] 
These changes in technology have changed the nature of 


management within manufacturing companies. Hawkins and 


oa, 


Barclay refer to these changes as the old and new orders of 
management. Table 2 summarizes the differences between the 
"Old and New" orders. 

TABLE 2 


OLD AND NEW ORDERS OF MANAGEMENT 


Old Order 


Production oriented | Market oriented 
Efficiency based Entrepreneur based 


Authoritarian "Co-operative" 
Stable structures Changing structures 
| Functionally based Inter-duseiplinary 
teams 





As can be seen from the table, the main difference between the 
two is that the new order of management is focused more toward 


controlling risk and increasing responsiveness to the 


Customer . 

In addition to these differences in management, 
engineers must change also. Engineers of the future must be 
"Commercial engineers" with skills such as: cOntwace 


negotiating and closing, selling and marketing, and persuasion 
and leadership qualities. The Finniston Report 
[Retard agrees with Hawkins and Barclay in 
stressing the need for future engineers to acquire and use 
these commercial skills. In order for tomorrow’s engineers to 
become proficient under the "new order" style of management, 
it is essential that they learn and practice the commercially 


based skills. In a survey conducted by Hawkins and Barclay 
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[Ref. 13, p. 50), employers were asked what qualities were 
considered most important when promoting engineering graduates 


to management positions. The responses are shown in Table 3. 


TABLE 3 


FACTORS CONSIDERED FOR MANAGEMENT PROMOTION 


Percentage 
Bactox Rating 


People Management 
Communication Skills 
feacia Record 


Interpersonal Skills 
Technical Awareness 
Project Leadership 
Personality 

Preetit/ Bette rency 
Innovative Skills 
Training Undergone 





As can be seen from the table, people-related skills are 
considered to be among the most important, whereas 
innovative type skills were lower on the list. 

Another factor to be considered when implementing 
training is the importance of timing. If management training 
is given too early in an individual’s career it will be 
difficult for them to understand some of the concepts and 
ideas. On the other hand, if management training is given too 
late, when the individual already has a large staff, there is 
little chance to experiment in developing one’s own management 
style. The balance between being there long enough and not 


being there too long must be found. According to Hawkins and 


17 


Barclay, this balance can be found "when the individual is in 
charge of a small number of people." It is at this time that 
the new managers are more peer oriented and have more support 
from their peers, allowing them to experiment with the new 
knowledge, skills, attitudes, and behavior that they’ve 
attained. [Ref leper) 0) 

The last point to be brought up about management 
training is that in the year 2000 it is estimated that, @oeea 
company’s current workforce, 70 percent of that workforce will 
be employed by the same company for whom they currently work. 
This implies that in order to ensure adequate numbers of 
potential managers, attention must be given to developing the 
existing personnel. {Ret . 1372p steo 4 

A. SUMMARY 

This section has provided an overview on management 
training. The first subject referred to was in-house 
asaaens iG. It was determined that organizations, through 
proper in-house training, can develop within their employees 
the desired attitudes, culture, and techniques needed to 
accomplish their routine day-to-day tasks. However, for the 
in-house training program to be effective, three phases should 
be followed: planning, conducting, and evaluating. In-house 
training, according to Donald Kirkpatrick, “Gan Cceonsismaie. 


reading programs, on-the-job coaching, and classroom programs. 
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The second subject referred to was the future of 
management training. It was pointed out that management 
development programs are moving away from the 
technical/innovative aspect of training and toward a more 
people-oriented type of training. Until the schools can be 
taught to teach these kind of "soft" skills, more importance 
will be placed on in-house training. 

With more importance being given to in-house training, 
more attention will have to be given toward evaluating those 


courses. 


.. EVALUATION OF TRAINING 

1. Evaluation Measures 
Training directors need to know if the employees are 
learning from the training they are receiving. If they are 
learning, the training directors need to know if the employees 
are applying it to their jobs. The bottom line is this; is 
the organization receiving an equitable return on its training 
investment? To find out, training or development programs 
should be evaluated. There are two basic kinds of evaluation 
measures: those collected in or at the end of training and 
actual performance measures collected when the trainee has 
returned to the job. The former are easier to get but the 
latter are more valid. Trainees may say that they enjoyed the 


training and learned a lot, but the true test is whether their 


ig 


job performance is better after their training than before. 
[Retiael, p.eso4] 
2. Criteria used to Measure Training 
The goal of training evaluation is to determine the 
degree to which the basic relationship between training and 
job performance is accomplished. In order to determine 
whether or not training was successful, Kirkpatrick identified 
four levels of criteria that should be used when evaluating 
supervisory training programs. They are: reaction, learning, 
behavior, and results. [Ref. 15] The four Critter 
will be discussed below to help clarify the meaning of each, 
their relationship with each other, and the guidelines and 
procedures for using them. 
a. Reaction 
Reaction can be defined as how well the trainees 
liked a particular training program. Evaluation in terms of 
reaction is the same as measuring the feelings of the 
participants. It is simply a measurement in determining how 
Satisfied the participants are with the training they have 
received. When measuring a participant’s reaction to 
training, honest and meaningful answers are desired. TO 
obtain candid responses, the following five guidelines should 


be followed: 


1. "Determine whiaGyyou want FO finamour. 
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ay Use a written comment sheet covering those items 
determined in step one. 


3. Design the form so that the reactions can be tabulated 
and quantified. 


A. Encourage the participants to write in additional 
comments not covered by the questions designed in step three. 


5. Obtain honest reactions by making the forms anonymous. 


Even after following the above guidelines, there is 
still no assurance that any learning has taken place because 
measuring reaction does not include a measurement of learning. 

b. Learning 

Learning is defined as the extent to which the 
trainees have learned the information and skills presented 
during the training course. To establish a procedure for 
measuring the amount of learning that takes place in a course 
of instruction, several guidelines should be followed: 

1. A before-and-after test should be used to ensure that 
any learning is a result of the program. 


2. The learning should be measured on an objective basis so 
that quantitative results can be determined. 


3. Where practical, a control group, a group not receiving 
the training, should be compared with the experimental group 
that receives the training. 

These guidelines imply that evaluation in terms of 


learning is much more difficult than evaluation in terms of 


meaACeE LON : 
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c. Behavior 

Behavior is basically the extent to which on-the- 
job behavior of the participant has changed as a result of the 
training received. Robert Katz, a Professor at Dartmouth, 
wrote an article in 1956 for the Harvard Business Review 
entitled, "Human Relation Skills Can Be  £Sharpened." 
Karat rae [Res 22 ake) quotes Katz as_- saying, "Tl oe! 
person is going to change their job behavior, five basic 
reguirements must exist": 1) They must want to improve, 2) 
they must recognize their own weaknesses, 3) they must work 
in a permissive climate, and 4) they must have an opportunity 
to try out the new ideas. Without these four requirements 
being met, it will be difficult, if not impossible for the 
participants to change their job behavior. 

How do we prove that their behavior has changed? 
Kirkpatrick identifies four guidelines that should help: 

1. Measure behavior on a before-the-program and after-the- 

program basis. 


2. Allow enough time between the program and the after-the- 
program evaluation to allow for change in behavior to occur. 


oF Get information about the participant from as many 
sources as possible. 


4. If practical, use a control group to compare with the 
experimental group. This will help to prove that any changes 
in behavior were due to the program and not to other factors 
such as salary adjustments, coaching from the boss, or 
influence from other people inside or outside the 
organization. 
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There will be some difficulties associated with the 
above guidelines. First of all, measuring behavior before the 
Puodram, Steep ens, 15 A Gitficuler task: When measuring 
behavior before the program, the measurement would have to be 
taken enough times to be sure that it was not only accurate, 
but that the behavior observed was typical as well. Secondly, 
concerning step two, how much time between the program and the 
evaluation is enough time? The time may vary for different 
people and different types of skills so it may have to be 
measured several times to be sure that the measurement is 
accurate. As can be seen, evaluating training programs in 
terms of on-the-job behavior will be more difficult than 
measuring either reaction or learning. 

d. Results 

Results are the outcome of the training program. 
Bre -ould anclude such things as improved productivity, better 
quality, lower costs, meeting deadlines, more competition, 
reduced accidents, improved morale, lower turnover, and 
ultimately, more profits or better service." [{Ref. 15, p. 6] 

Determining what final results were accomplished due to the 
training program is the most difficult part of evaluating 
results. There are so many other factors involved that it is 
frequently impossible to prove that the training program 


caused the desired result. Again, as with the three previous 
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criteria, Kirkpatrick has developed the following guidelines 
to help in evaluating results: 

1. Measure the conditions before the program and compare 
with the conditions after the program. 

2. Use control groups to try to eliminate othermetfactone 
that could have caused changes in the results. 

As we look at the evaluation of reaction, learning, 
behavior, and results, we can see that it is much easiermes 
say that the training program is having a positive effect on 
the vabovemcriteria than Jt aS 70) picomemre- In some cases, 
proof is impractical and almost impossible to get. If this is 
the case, what do you do? According to Kirkpatrick ;eiigges 
shoot for the proof but be satisfied with the evidence." 
(Rete) 

Jackson and Kulp [{Ref. 18] suggest that it 
is better to approximate evaluation than neglect it all 
together due to its difficulties. For example, if you Game 
get measures of behaviors and results, then measuring 
learning, both at the end of the course as well as some time 
later, is better than doing nothing at all. 

3. Evaluation Techniques 


In his article, "A Training-Appropriations Procesa@® 


David L. Hobbs identifies two evaluation techniques: 
interviews and surveys. [Ref. 19] ACCORGING =O 
Hobbs, "An advantage of interviews and surveys is their focus 
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One 2s isRpeaviduad |. Respondents are not reacting to or 
influenced by the opinions of others. They are more free to 
express their own ideas and are more candid and specific in 
their comments. The interviewer or surveyor benefits from 
having a high level of control over the direction and depth of 
phe? dinquiry." peed elo; Poss LO) Table 4 identifies 
additional advantages as well as some disadvantages associated 


with administering a questionnaire. {Ref. 18, p. 28] 


TABLE A 


ADVANTAGES AND DISADVANTAGES 
OF QUESTIONNAIRES 


Advantages Disadvantages 


— ae - em ee 


1. Can usually be 1. Can obtain only estimates 
administered to a large number | of what people actually think, 
of people, with less time, and | feel, do, or produce. 

money than most other 

strategies. 


———— ————_———_ em a ee —- _— ep = — — = ~ -_ 


2. Ensures that everyone 2. Responses may be biased be 
responds to identical respondents’ desire to present 
questions/directions. themselves in the best 
possible Jight-or to respond 
in a way that they think will 
|; please the author or recipient 
of the questionnaire. 





When using a questionnaire, you must ensure that it is 
valid and reliable. If the questionnaire measures what it is 
Supposed to measure then it is considered valid and if it 
measures consistently then it is reliable. To determine if a 


questionnaire is valid to the area of study, its questions are 
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compared with the program objectives. If each objective is 
covered by at least one question, the questionnaire meets the 
basic criterion of valerie, [Ref . "204 
A. Contingency Variables 
Instead of measuring variables before and after the 
training course or using control groups to see if any 
improvement occurred as a result of the training, Clement and 
Aranda decided to look at variables beyond the training 
course. fRef. 21] These variables, referred to as 
"contingency" variables, include the organizational setting 
within which the manager attempts to use the training, the 
mature of the manager to be trained, and the nature of the 
problem to be solved through training. 
a. Organizational Influence 
Robert House [Ref. 22] identified three 
factors that affect the transfer of training from the 
classroom to the job: the formal authority system within the 
organization; the immediate superior of the trainee; and the 
primary work group of the trainee. The first factor refers to 
the possibility that top management has established company 
policies and objectives. Any training that does not comply 
with those pre-existing standards will not succeed. The 
second factor, the supervisor, has a tremendous impact as to 
whether or not the trainee implements any of the new ideas. 


If the immediate supervisor does not agree with what the 


Z6 


trainee was taught, and that supervisor controls the rewards 
and punishments, then the trainee is not likely to use what he 
or she has learned. The last factor, the trainee’s primary 
work group, refers to the possibility that the trainee may 
succumb to the expectations or pressures from his peers and 
subordinates, which may be inconsistent with training. 

All three of the factors listed above can influence 
a manager’s attempt to use the concepts learned in a training 
course. To ensure that the factors have a positive effect on 
the trainee, all courses taught should reinforce the policies 
and practices of upper management and all personnel should be 
indoctrinated as to what upper management expects of them. 

b. The Nature of the Manager Being Trained 

All managers differ in their experience, education, 
abilities, attitudes, and values. Therefore, the training 
received by each individual, although the same, will have 
different effects on each person. To determine the change in 
Supervisory attitudes, researchers Hayes and Williams, ina 
1971 study, found that attitude change is inversely related to 
age, seniority, and span of control. [Ref. 23] The 
study concluded that: "supervisory training programs are more 
effective when the participants are young, relatively new to 
supervisory ranks, responsible to a small number of 
subordinate personnel, and have a short period of total 


service." 


a 


c. The Problem to be Solved Through Training 

This contingency variable is concerned with 
matching the proper training method to the desired training 
objective. A recent survey of training directors has found 
that case studies are best for the development of problem- 
solving skills; programmed instruction for retention of 
knowledge; and both role playing and sensitivity training for 
changing attitudes and improving interpersonal skills. 
[Ref. 24] The success or failure of a particular 
training program could rest with the type of training 


technique used. 


cc EVALUATION OF MANAGEMENT TRAINING 

Adequate evaluation of training in organizations is rarely 
done. (Reps a4 Many articles have been written on 
the topic, but most address the difficulties inherent in the 
process. For example, negligence on the part of management in 
defining appropriate evaluation criteria, the difficulty of 
establishing scientifically matched control groups in field 
research settings, and that small numbers of trainees makes 
hypothesis testing and inferential statistical analysis 
Gifficult. Additionally, good evaluation is time consuming 
and expensive; and to compound the difficulties, training 
administrators cannot come to an agreement on a definition of 


"good" training evaluation. [Ref. 26] 
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The evaluation of management training has its own set of 
inherent problems. Oneepon- close 1S “the “difiveulty ‘an 
collecting data on “hard” criteria. Tnacs. Lack “Or (Ward! 
criteria is not unusual in the world of management training 
where expected outcomes are often not stated or, when 
specified, are hard to measure. Jackson and Kulp have 
identified through their research many differences between 
management and technical performance that might influence 
evaluation. [Ref. 18, p. 5] These differences are: 

- The nature of management training makes it harder to 
describe, observe, and measure outcomes as compared to 


technical training. 


- Performance standards for management positions are less 
likely to be defined or agreed upon. 


- Management positions tend to deal more with qualitative 
problem-solving situations whereas technical problems tend 


to be more concrete. 


- Management outputs are harder to judge for quality and 
often times they are not related to specific tasks. 


- Management performance often requires responding to 
unpredictable, unscheduled events. 


Jackson and Kulp have identified other differences between 
management and technical performance, however, they felt that 
those listed above were the reasons most likely to influence 


the evaluation of management training. 


D. SUMMARY OF LITERATURE REVIEW 
To summarize, the literature reviewed consisted of three 


main topics: management training, the evaluation of training, 


Zo 


and specific difficulties involved in evaluating management 
trang. Within the first topic, it was determined that 
future managers will have to learn a new style of management-- 
away from the production oriented, authoritarian style--toward 
a more market oriented, cooperative style. Additionally, 
engineers who want to become managers will have to be more 
people oriented vice technically innovative. To help 
accomplish this, in-house training should be used. In-house 
programs allow the organization to develop within the 
employees the attitudes and values that it feels are most 
IMpere ant. 

Within the second topic, the evaluation of training, four 
criteria were developed to help measure the effectiveness of 
training. The four criteria used were: reaction, learning, 
behavior, and results. These four variables are incorporated 
before and after evaluations, as well as with control groups, 
to help measure whether or not the course was successful. 
Although it is difficult to meet these standards to prove what 
is successful because of training and what is successful 
because of outside circumstances, it is still better to 
attempt to determine the effectiveness of a training course 
than to do nothing at all. 

Finally, it was determined that management training is not 
evaluated as often as technical or clerical training. Of the 
several reasons listed above, the overriding explanation for 


this is the fact that most management training courses do not 
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efate theixz training ob ectives. Those organizations that do 
state their training objectives, however, do not state them is 


measurable terms. These two conditions, makes evaluation of 


Menmeaqement training Gifficult 1£ not ampossible. 
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III. METHODOLOGY 


A. GENERAL 

This chapter focuses on the method used in the development 
and execution of the questionnaire which was used as the 
primary data gathering instrument for this research. Specific 
data about NAC’s management training program were collected 
through the use of this questionnaire. Section B discusses 
the development of the questionnaire. This section describes 
the survey questions that were designed to determine whether 
or not the individual’s job performance is different wace. 
result of the management training received. Section =e 
discusses how the survey sample was determined. Section D 
discusses how the data were collected. And lastly, Section E 
provides recommendations pertaining to this and subsequent 


questionnaire development. 


B. DEVELOPMENT OF THE QUESTIONNAIRE 

As identified in Chapter II, there are four criteria used 
to measure management training: reaction, learning, behavior, 
and results. [Ref. 18] Due to the time constraints of this 
study, the only criterion we were able to evaluate was 
peace 1on. This was accomplished through the use of a 
questionnaire. The purpose of the questionnaire was fourfold: 


(1) to determine the relationship between management training 
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and job requirements, (2) to determine the satisfaction of 
trainees with specific management courses, (3) to determine 
opinions on the relationship between management training 
received and changes in job performance, and (4) to determine 
if it is perceived that NAC benefits from the management 
training the trainees receive. 

The questionnaire consists of a combination of closed and 
open-ended questions. However, to get the best response to 
the questions, it was determined that the majority of the 
guestions should be open-ended vice closed. Open-ended 
questions allow respondents to answer freely in their own 
words and they can be worded in such a way that they either 
ask the respondent a broad general question or a relatively 
specific question. Table 5 identifies some additional 
advantages as well as some disadvantages of administering 
open-ended style questions. [Re terse np e272. | 

Closed questions, on the other hand, are best used for 
demographic type questions and questions that don’t require 
a specific explanation. Closed questions present respondents 
with possible responses from which they choose the most 
appropriate one(s). This type of question can include a wide 
variety of responses, including multiple choice, matching, 
rating, ranking, true-false, or yes-no. Table 6 identifies 
both the advantages and disadvantages to using closed 


questions. [Ret 18 ;-S] 22) 
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TABLE 5 


ADVANTAGES AND DISADVANTAGES 
OF OPEN-ENDED QUESTIONS 


Advantages Disadvantages 


= ——————— 
F 










1. Questions can usually be 1. Questions may be difficult 


developed easily and quickly. to score and analyze because 
of the variability of the 


responses. 


2. Questions allow a free |} 2. Questions usually take 

expression and may pick up more time and effort to 

information that would be complete than closed; result 
}missed in a closed question. may be lower response rate if 


respondents can choose MeE to 
respond-or in brief, 

uninformative responses where 
there is no perceived choice. 


TABLE 6 


ADVANTAGES AND DISADVANTAGES 
OF CLOSED QUESTIONS 


Advantages Disadvantages 


hs 2 eerie = ————————— 






1. Can usually be scored and 1. May miss important 
the results analyzed quickly information since they limit 
| and easily. respondents’ options. 













12. Can usually be completed 
quickly and easily and may 
result in a higher response 
rate. 


The questions asked on the questionnaire were designed 


after an initial trip was made to NAC in which 12 engineers 


from randomly selected departments were interviewed. The 
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information obtained from the interviewees enabled the 
researchers to develop a questionnaire that was custom 
designed for the scientists and engineers employed at NAC. 
The research questions, identified in Chapter I, can each be 
linked to one or two specific questions contained within the 
questionnaire. Research questions number 1 and 2 are answered 
by survey questions 7 and 8a, and 8d and 9 respectively; 
research question 3 is answered by survey questions 4, 5, and 
8b; and lastly, research question 4 is answered by survey 
questions 6 and 8c. See Appendix C for a copy of the 
questionnaire used in this study. The questions in the 
survey were specifically designed to help evaluate NAC’s 
management training program. 

Question 8 from Section II confused some of the 
respondents. The question asked respondents to rank, from 1- 
5, how they feel about certain statements. A eo eo ae ies 
questionnaire, 1 stood for strongly disagree and 5 stood for 
strongly agree. After reading the respondents’ answers to 
previous questions within the survey, it was determined that 
4 out of the 75 respondents reversed the ordinal ranking and 
gave, for example, a one when they really meant to give a 
five. This problem is considered minor because no more than 
5 percent of the respondents committed this error and no other 


consistent error patterns were noted. 
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ce: SAMPLE 

NAC management identified a specific set of management 
courses that is offered primarily to its scientists and 
engineers. NAC feels that these 12 courses require special 
attention because, when compared to other management courses, 
these are not only more costly but they are attended more 
frequently as well. 

How was the determination made as to who would and who 
would not be sampled? In order to be chosen, the scientists 
and enqineers had only to meet one criterion; they had to have 
taken at least one of the management training courses 
identified by NAC. This criterion, extracted from “tees 
historical personnel data base, identified 239 individuals. 
However, because there is more than one research topic being 
conducted in this area of management training and training in 
general at NAC, the list of qualified participants had to be 
reduced. To eliminate "survey burnout," 28 engineers were 
dropped from consideration because they had participated ina 
Similar survey approximately two weeks earlier. An additional 
12 engineers were dropped because they had taken part in the 
initial interviews that determined what kind of questions 
needed to be included in the survey. Lastly, three more were 
excluded because two of the individuals had retired prior to 
administration of the questionnaire, and the third individual 
was erroneously identified as a qualified participant. In 


addition to the above 43 individuals, another 61 had to be 
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unexpectedly removed from consideration due to funding 
constraints for department 200. All eligible engineers from 
within that department had to be omitted. These reductions 
eventually pared the list of qualified engineers’) and 


Serencrsts down to 135. 


| a DATS. COLLECT LOM 

Initial plans called for all the survey participants to 
complete the questionnaire in a conference room on NAC 
premises. The participants were to be assigned into groups of 
30 and each group given 30 minute blocks of time to complete 
fhe £LOrm.. However, due to financial constraints, MAC could 
MPorealblow -the 135 participants to take the time off to 
participate in the study. As a result, the data collection 
for the questionnaire consisted of the following steps: 


1h The researcher visited NAC LOE the actual 
administration of the survey. 


2. 135 questionnaires were given to the Branch 
Seeretaries. for distribution . 


3. The participants were given two days to complete the 
questionnaires. 


4. At the end of the first day, only 13 questionnaires 
had been returned. 


5. During the second day, a reminder message was sent to 
each of the participants via electronic mail. A 
"received message” response was required. The message 
requested completion of the questionnaire by close of 
business. 


6. At the end of the second day, the researcher had only 


received an additional 29 questionnaires for a two day 
GOGAde sO b.i4.. 
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7. The following week, another 38 questionnaires arrived 
in the mail, bringing the total  “Uee to 80 
questionnaires received. 
For reasons of anonymity, and in the hopes of obtaining candid 
responses, names were not requested on the questionnaires. As 
a result, not only was there no way to ensure completion of 
the questionnaires but there was no way to find out who did 
and who did not return them as well. The final response rate 
was only 59 percent (80 responses out of 135 surveys). 

After analyzing the 80 returned questionnaires, three were 
rejected because the individuals had critiqued the wrong 
course and two more were rejected because the trainees did not 
attend the class. The analysis was performed with the 75 
remaining questionnaires; Chapter IV contains the results 


obtained from those questionnaires. 


E. RECOMMENDATIONS 

The recommendations provided here concern the distribution 
of the questionnaire and are not related to the conclusions 
and recommendations presented in Chapter V. 

Upon completion of the survey, one problem area was 
encountered pertaining 1S the distrib aem of the 
questionnaire. If at all possible, the individuals involved 
should be administered the questionnaire in a controlled 
environment. The definition for controlled environment in 


this case means two things: (1) the researcher should go to 


the command/organization where the questionnaire will be 
administered, and (2) the participants need to be gathered 
together ina single location; either in one large group or in 
many small groups. 

There are two reasons why the researcher should opt for a 
controlled environment. The first reason is to get a higher 
response rate. The simple fact is, a "captive" audience is 
more likely to respond to the questionnaire. Additionally, 
employees perceive the survey as important because work time 
is provided for the questionnaire to be completed. 
[fRet. 27] Secondly 7 the researcher ae the 
questionnaire is available should any questions arise. This 
is important because it will ensure that any confusion about 
the questionnaire will be clarified and better survey data 


will be generated. 
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IV. RESULTS 


A. OVERVIEW 

The primary focus of this research is to determine: Cis} 
if the trainees are receiving management training that is 
relevant to their job requirements; (2) if the trainees are 
satisfied with the specific management courses; (3) if there 
is a perceived relationship between the management training 
received and changes in job performance; and (4) if there are 
any benefits that NAC receives as a result of the management 
IE sefeibig alin (oee In answering these four points, this chapter 
presents a synopsis of the responses based on the data 
obtained from the questionnaires. These responses only 
concern the data received from the twelve management courses 
that NAC wanted examined. The remainder of this chapter 
consists of two sections. Section B is an analysis of the 
demographics of the scientists and engineers who responded to 
the questionnaire and Section C provides the results obtained 


from the survey data and conclusions. 


Be SURVEY RESPONDENTS 

As noted in the last chapter, the number of NAC employees 
who had taken the management courses of interest in the last 
year and a half was small. The number decreased even further 


due to the methodological difficulties described. As a 
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result, the validity of these results can be questioned. 
Under these circumstances, a characterization of the survey 
respondents is appropriate. By comparing respondents to the 
population of all who could have responded, and to the 
population of all NAC scientists and engineers, information is 
added about the representativeness of these data. These 
comparisons are made below. 

Table 7 identifies, by course, the percentage of surveys 
returned compared to how many people had actually taken the 
course. The pubpose of ~Table 7 1s. to point, out. that “che 
Gomelusions being drawn from the data may not be 
representative of the population of all those who have taken 
these courses. Although four of the twelve courses obtained 
Sueceturhn crate Of 50 “pencentr or higher,~ they are Still. not 
practically significant because the initial number of eligible 
participants was quite small. Additionally, the first three 
courses, Basic Project Management, Dale Carnegie, and 
Performance Feedback all had a large number of eligible 
participants but the response rate to the questionnaire was 
very low. With these limitations, these data should be 
considered exploratory in nature. 

The data obtained from the questionnaire will provide to 
NAC a profile of the type of personnel who responded to the 
management training questionnaire. The specific areas of 


interest are: gender, years worked at NAC, paygrade, 


4] 


TABLE 7 


PERCENTAGE OF PERSONNEL ELIGIBLE 
TO RESPOND COMPARED TO THOSE 
THAT ACTUALLY RESPONDED 


ELIGIBLE ACTUALLY 
COURSE TO RESPOND RESPONDED | PERCENT 


Basic Pt ogece 
Management 
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Dale Carnegie Effective 
Speaking 


Performance Feedback 
Power of Effective 
Listening 

Train the Trainer 
Managerial 
Communications Skills 


N 
ce) 
a) 
Oo 
W 
is 


Un bh [4 No bs ul W [4 
ro) iS ro) Ne) = a ron » 


Ss) 









Supervisory Institute 
Dealing with Difficult 
People 

Valuing Diversity 
Assertiveness Skills 


Teambuilding 


Statistical Process 
Control 


department currently working in, and lastly, supervisory 


on 


pnd 
<> 


i) a 1) 
NO ~J > 1 ~J ~J nas OY 


Oo 


experience. 
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The first area to be considered is gender. According to 
the data, 8 percent of the scientists and engineers who 
answered the questionnaire were females and 92 percent were 
males. Within the organization as a whole, women represent 13 
percent of the scientists and engineers. 

As anticipated, the majority of the trainees who 
responded, 61 percent, have worked at NAC for ten years or 
less and 83 percent have not been formally designated as a 
supervisor. This makes sense because management training is 
intended mostly for those employees who are inexperienced in 
supervisory skills and relatively new to the organization. 
[Ref. 13] 

Related to the two demographics above is the third area of 
consideration-~paygrade. There is a direct relationship 
between an individual’s paygrade, number of years worked, and 
supervisory experience. In other words, if the individual is 
a GS-11, chances are that the trainee is not a supervisor and 
has only been at NAC for 10 years or less. The data show that 
74 percent of the trainees who participated in the study are 
between the grades of GS-9 and GS-12; within the NAC 
organization as a whole, 77 percent of the scientists and 
engineers are between those two grades. 

The distribution of research participants by department 
shows that departments 800 and 900, together, make up 
approximately 56 percent of the scientists and engineers 


within NAC. The next closest department, department 200, only 
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makes up 15 percent. The survey respondents from departments 
800 and 900 accounted for 81 percent of the study 


participants. 


C. ANALYSIS 

Since Basic Project Management and Dale Carnegie received 
response rates of over 30 percent, they will be analyzed in 
detail. The analysis of those courses addresses the four 
research questions identified at the beginning of this 
ehapiere The remaining courses are analyzed as a group and 
will not specifically answer the four research questions. For 
both the detailed analysis and the group analysis, the 
numerical scores given to questions 8a-8d will be totaled 
individually by course. That total will then be divided by 


the number of respondents from each course and reported in the 


form of an average score. Question 8 was rated on a scale 
from 1-5 where one is "strongly disagree" and five is 
"strongly agree." An average score of five is considered a 


perfect score and corresponds directly to "strongly agree." 
In analyzing the open-ended questions--questions 4, 5, and 
6--a content analysis was conducted. As part of the analysis, 
the major themes, given by the respondents, were drawn from 
each question. The answers were then grouped according to 


theme. 
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1. Objective One 
The first objective of this study is to determine if 
the trainees are receiving management training that is 
relevant to their job requirements. This question can be 


mapped to survey questions 7 and 8a. 


Question — 7: Have you had any difficulty in applying 
anything you learned in the course to your job? If yes, why? 


Course: Basic Project Management 

N x 
ETE years, ac a dee el an aa ae ea a eer Re A Te Ze 58 
16) I es Se ee ee ee ee eer ee ae L6 42 

Ge OF yes responses: 
IOAN ieore Use mis ma OO oer ean 8 5) 0 
Zp Ne, tesa ne Cae Le Me nit aia epee tance) 6 38 
BS) 1a Ga Cg angen ek a een eg cn Z 152 


For this course, almost half of the respondents said that they 
did have difficulty in applying what they learned to their 
job. The two most frequently mentioned reasons were that the 
course was not applicable to their current job and that they 


were too busy to try the new process. 





Course: Dale Carnegie Effective Speaking 
N % 
UM EY) teers cae Sega Is ae rela abe oe acre bare 8 80 
1S UMS Re Rs Wea gana ne a eer gay eae aa a Z ZU 
c. Of yes responses: 
eee ee et: Ort LeMen t... . ces s < «fe 2 100 


Although this course has few responses, they do represent 34 
percent of the population that has taken the class. What is 
significant to note here is that 80 percent of the respondents 
indicated that they did not have trouble implementing what 


they learned to their jobs. 
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Question 8a: This course was relevant to my job, 
i.e., it addressed problems and issues I encounter in my job. 


Course Avg. 
Basic Project Management a (BEM) eee 3a 
Dale Carnegie Effective Speakingm@(DC) -3. 2. 32... 4.20 


The results show that the trainees thought that the DC course 
was more relevant to their jobs than the BPM course. 
The extreme numbers that make up the averages for the BPM and 
DC courses are: 3 ones and 13 fives for BPM and 1 one and 6 
faves for DC. It should be pointed out however that the one 
that was given to the DC course, in all likelihood, should 
have been a five. After reviewing all of this respondent’s 
answers, it was felt that he reversed the scale for this 
question because the answers given were not consistent with 
the rest of his responses to the survey. With a five instead 
of a one, the average for this question increases to 4.60. 
This problem with the reversed scale will persist for the Dale 
Carnegie course throughout question number 8. 
2. Objective Two 

The second objective of the study is to try to 
determine if the trainees are satisfied with the specific 
management courses. To help achieve this objective, survey 


questions 8d and 9 were developed. 
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Question 8d: Overall, I was satisfied with this course. 


Course Avq. 
Basvemeeouece, Mamagememin. «524 nc acu tas sue we eee 4.03 
bale tearneqve BEreclive Speacind ois... 2 Mabe 2 Aid 


The averages here indicate that the participants agreed with 
the statement. However, the participants of the DC course 
agreed somewhat more strongly because there was not one 
instance of a rating below four. With the reversed scale 


problem corrected, the average for DC would be 4.7. 


Question QY: Sheu NAC COnt nue oO Ooffer-Chis scource: 


Course N_ 6 Yes 
Basic Project Management (BPM)............. 35 20 
Dale Carnegie Effective Speaking (DC)...... ile 8) 100 
This question and question number 8d are very Similar. With 


high averages received for question 8d, it was anticipated 
that high percentages would follow for this question. FOr 
BPM, only one person out of 38 answered this question 
negatively. His complaint was that the course did not teach 
him how to manage a project. All of the respondents of the DC 
course indicated that NAC should continue to offer this 
course. 
3. Objective Three 
The purpose of this objective is to determine 

1f there 1s a perceived relationship between the management 

training received and changes in job performance. This 


objective is related to questions 4, 5, and 8b in the survey. 
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Question 4: Describe any non-observable ways you have 
changed as _ a result of this training, for example, inion 
thinking, level of understanding, attitude, etc. 


Course: Basic Project Management 

N_ a 
a. Increased understanding. ] ee) . see, Ales 45 

b. Better time utilization through 

planning and organization, e.g., 
Cut. down On Overt ae ae ee ical 29 
OPM \(e) ol Perma rer rer ey EC Se oF 6 16 

d. Misc (Improved attitude, Better 
customer relatwvons) (te. eee 4 ibe: 


After content analyzing the responses to this question, it was 
determined that 84 percent of the participants reported 
positive non-observable changes whereas only 16 percent of the 
participants reported no non-observable changes. Of those 
that reported gains, 66 percent reported better understanding 
and improved attitude and 34 percent reported improved results 
on the job. Of the six respondents that reported no non- 
observable changes to this question, two were "directed" to 
take the course and (probably as a result) reported negatively 


on nearly every question in the survey. 


Course: Dale Carnegie EFfective Speaking 
iN S_ 

a. More open-minded, improved 

attztude, accepting sot other 

VILEWPOLN WS so Gages sens ose eee ee 6 60 
b. Improved communications/interpersonal 

SkKALLS ole ee oko 5 Se cc, Zz ZG 
c. Improved self-contidence 2a. ee ee 2 20 
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All ten participants reported positive non-observable changes 
eeea result “Of the Vtiraiming. Sixty percent of the 
participants felt that the course improved their attitudes or 


that they had become more open-minded. 


Question 5: Describe any behavioral ways you have changed 
as a result of this training: what do you do differently now 
Om vOur 70Ob as a  .s,esult_of having taken this course? 





Course: Basic Project Management 

9 
SL See rennoroans 2eq sty IertEeG 255 sitet ae eo 39 
Ley MemG Eh a a eA o sees, CIE os. 5, Cah nasa ename oie ites Gee 10 26 
Cg == 0G) gm \ 0 ea oF1bs= 8 01g wl glia fregranteantae = aera, Pama ae Nay ere ade ener 2 Nek 

day Eneeqracion (of ©"big picture’ data, 
MOmwe WESSON dee aes. = Weeee fa eee et tite eae gate cue aes 4 Al 
e. Better at customer/peer relations.......... 2 S 
£2 “Too busy “to apply Wheat was. learneds )«...42% it S 
fr eam lh colar an ne Se eRe oe ea re eer pera ere Z is, 


Seventy one percent of the trainees reported positive changes 
as a result of the training they received, whereas only 29 
percent indicated negative or no changes. Five of the ten 
participants that indicated no change, were the same 
individuals that indicated no change for the previous 
question. However, in this case, only one of the participants 


was "directed" to take the course. 


Course: Dale Carnegie Effective Speaking 
ae Better et eworking w/ people... ..ccstee se eee = or 
Dee SCeeCEyPGeseMeat OMS. cin. = 44 4 \4 .¢ iemeruuenueae Z, dle 
Beer PIO tek Sas eta eee aor, i his ai Desa cknts, & bse temas eee Z a) 


ol. More, positawve...<...25 «2. ee au 8 


For this question, a total of 12 responses was received from 


ten participants, i.e., at least one of the participants made 
more than one response to the question. As can be seen from 
the above responses, all trainees reported a positive 
improvement in their behavior after the training. At 58 


percent, working better with people was the response most 


often mentioned. 


Question 8b: This course met the objectives of 
improving my abilities in this area. 


Course Avg. 
Basic Project. Management. =.= 3..5).... eee 3398 
Dale Carnegie Effective speaking. - 7... eee ee 4.20 


The average for BPM is the lowest received thus far. However, 
the participants still leaned more toward agreeing with the 
statement than toward being neutral. The high average 
received for the DC course indicates that almost the entire 
Sample strongly agreed with the statement. The actual numbers 
for DC were 9 fives and 1 four. This average takes into 
account the reversed scale problem. 
A. Objective Four 

The purpose of this question is to determine if there 
are any benefits that NAC receives as a result of the 
management training. The data received from survey questions 


6 and 8c provide data relevant to this question. Question 6 
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asks the respondent to consider how his/her observable and 


non-observable changes have benefitted NAC, if at all. 


Question 6: Please consider your answers to questions 4 


and 5 and describe how you think these training outcomes have 
benefitted NAC. 


Course: Basic Project Management 

nN _ 
ee aa On ledge @ ie) Se te Bere 1. e5 a ate ror eae oe 18 47] 

b. Improved response time, more 
Comp eke OM ¢ 2 AsO WS five Os © ace geg sates te ee a os aa 2 24 
@ Betber Comminicatiton/ Coordination... «h.«<. iy 18 
@= etter Manager /endineer 6420.22 2. cee oe 6 16 
5 WEE OS 0c). Rae re ee eee ee ea ee ae ee ee 6 iG 


Sixteen percent of the respondents said that NAC did not 
benefit from the training they had taken. The rest of the 
participants indicated that NAC did, in fact, benefit from the 
training they received. Of the remaining 84 percent, which 
consisted of 40 responses by 36 people, 45 percent said that 
NAC was more efficient as a result of the training received. 
Additionally, 22 percent said that the customers of NAC were 
receiving the benefit in the form of lower costs and improved 
response time. An Interesting note is that there was a least 
one person in each of the first three categories that reported 
no observable benefits in question 5, yet they still felt that 
NAC benefitted from them taking the training. Additionally, 
one respondent who was extremely satisfied with the course 
felt that the course should not be recommended for new 


engineers or technicians. This question received eight more 


aye 


responses than it had participants. This is due to several of 


the participants who gave multiple responses to the question. 





Course: Dale Carnegie Effective Speaking 
N_ % 
a. Better representation of NAC 
With SPOnSOD .. 00 wee. - >... eee. 2 4 40 
b. Working better with other 
GMP lL OY SES eis os greets eee ete ere 7 70 
one vb hon rer or ee ee il jn8 


Once again, as has been the case with the DC course, a high 
percentage (in this case 100 percent) of the trainees 
indicated that their training has benefitted NAC in a positive 
manner. As indicated above, this course also received more 
responses than participants and this is also the result of 


multiple responses from two participants. 


Question. Ge: NAC has benefitted from the management 
training I received. 


Course Avg. 
Basic Project, Management ~ « .qcue esis eee ae > eee 3 ee 
Dale Carnegie Effective Speaking.......... /..).ee 4.80 


This question asks for the same information as number 6. The 
redundancy was built into the question to see if the 
respondents were being consistent with their answers. Upon 
comparing the two questions, it was determined that the 
respondents were being consistent. A somewhat lower positive 
response rate for both questions 6 and 8c was recorded for 
BEM: Whereas, an almost perfect positive response rate was 
recordedmrorn Ue. For the DC average, the reversed scale 


problem has already been adjusted. 
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>. Summary 
a. Basic Project Management 

The data revealed that 100 percent of the engineers 
taking this course had not been formally designated as a 
supervisor. This includes those who were not designated and 
had not supervised other persons during the past year (58%), 
and those who were not designated but had supervised other 
persons during the past year (42%). Additionally, 95 percent 
of the engineers who took this course were either GS-lls or 


ve 


9) 


Results of the course content evaluation show that 
a2 percent of Ehe eGngineers are having difficulty applying 
what they learned in the class to their job. However, in 
Spite OL that difficulty, 37 of the 38 respondents indicated 
midis NAC Should continue to offer this course. This number 
includes five individuals who said that they were "directed" 
to take the course. A possible reason for this disparity is 
that although the material being presented is applicable to 
their jobs, the trainees are not yet in the position where 
they can put to use what they learned. The reason given by 
the one person who thought that the course should not be 
continued at NAC was that it did not teach the trainees how to 
manage a project. However, this one negative statement is 


countered by approximately 37 other positive statements. 
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b. Dale Carnegie Effective Speaking 

The engineers that took this course had the same 
basic characteristics as the engineers that took Basic Project 
Management. Not only had 100 percent of these engineers not 
been formally designated as supervisors, but they were also 
all GS-lls or 12s. All ten respondents gave positive comments 
and indicated that the course should be continued. Many of 
those same trainees also indicated that this was the best 

training that they had received. 

6. Remaining Courses 
Since the remainder of the courses had such small 
response rates, they will be looked at concurrently and any 
significant findings will be identified. The first questions 
to be analyzed are questions 8a-d. To review, question 8a 
measured the relevance of the training to the job, 8b was 
designed to measure whether or not the training improved 
abilities in that specific area, 8c identified if NAC 
benefitted from the training, and 8d determined if the 
employees were satisfied with the course. Table 8 identifies 
the ten remaining courses as well as the average of the 
responses to each question. 

Other than Valuing Diversity, all the other courses have 
averages that indicate positive attitudes toward the training 
received. However, the last five courses only had one 


respondent and are therefore considered to have limited 
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TABLE 8 


AVERAGED RESPONSES TO QUESTION 8 


Performance Feedback ohn lcal 4.17 


POWer Of Bt rectas-c 
Listening 


Managerial 
Communication Skills 


| ee 
Institute 


Dealing w/ Difficult 
People 


Valuing Diversity 
Assertiveness Skills 


Teambuilding 





reliability. Overall, as can be seen from the combined 
averages in the last row of Table 8, the respondents felt that 
1) the courses were relevant to their jobs, 2) they improved 
their abilities in that area, 3) NAC benefitted from the 


training, and 4) satisfied with the course. 
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a. Performance Feedback 

The individuals who took this course were people 
who had worked at NAC for more than 11 years, were either a 
GM-13 or 14, and had been a designated supervisor from 
somewhere between one to ten years. Some of the positive 
gains reported from the course were that they are now better 
at anterviewingqg and that they provide feedback more 
frequently - Of the six respondents, two were required to 
attend and of those two, only one answered negatively to the 
questions. The negative respondent, when asked if NAC should 
continue to offer this course, Said no because he feltGwiiar 
the course should only be offered to new supervisors. 
However, both respondents were consistently responsible for 
aqiving the lowest scores as their answers to question number 
eight. Instead of strongly agreeing that the course was 
relevant to the job, these two respondents only agree with the 
Statement. Additionally, they were neutral as to the course 
improving their abilities in that area and to ther overage 
Satisfaction level. 

b. Power of Effective Listening 

Of the responding sample of participants who have 
taken this course, the majority of the personnel were GS-l2s 
and GM-13s who had been working at NAC for approximately 6-10 
years. The averages from the above table indicate a course 


that is only somewhat effective, receiving the lowest combined 


averages in three of the four questions (excluding Valuing 
Daversa ty) The respondents thought the course was quite 
relevant, were fairly positive that it improved their ability, 
felt less confident that NAC benefitted, and felt even less 
Satisfied. However, one of the survey respondents was 
required to take the course and it is the responses from that 
individual that brought down the averages. Additionally, all 
of the respondents, except the individual that was required to 
take the course, had positive comments about the training; 
aS eae improved concentration, better communication, and 
Improved efficiency. 
c. Train the Trainer 

Four engineers responded to this survey. All of 
their responses to the questions were positive except for one 
comment . One individual indicated that the course did not 
teach instruction methods, the primary purpose for taking the 
course. From the rest of the participants, a sampling of 
their responses are: the course gave them more confidence in 
their teaching skills, it made them more conscious of body 
language, and it benefitted NAC by making the participants 
more organized and more effective as instructors. The 
respondents are GS-12s and GM-13s that are not formally 
designated supervisors, and the majority of which, three out 


of four, have been employed at NAC for over 20 years. 


oT. 


d. Remaining Courses 
Of the seven remaining courses, six consisted of 
positive comments about training. However, due to the small 
number of responses received, no significant conclusions can 


be obtained that would be relevant to the analysis. 
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ae RECOMMENDATIONS AND SUMMARY 


2 RECOMMENDATIONS 

Based on the data analyzed in this study, the following 
recommendations are made: 

1) NAC should continue to systematically collect these 
same kinds of data both immediately after people have taken 
the course as well as 6-12 months later. 

2) NAC should design and implement a training evaluation 
program to allow for a more comprehensive review of the 
courses being taught--if not for all the courses, then at 


tease £or the courses with high through-put. and/or cost. 


3) NAC should conduct a cost benefit study of several of 
the courses evaluated in this study. 


1. Recommendation 1 

This recommendation will give NAC a more accurate 
picture of how representative these data are and if the 
employees truly value these particular courses. Thas is 
probably the simplest and least expensive method of evaluating 
training but it is also the least effective. Simply measuring 
whether or not an individual liked the training does not 
indicate whether anything was learned. This evaluation would 
measure reaction data, in part, but as with this study, it 
Should attempt to go further by assessing perceptions of 
learning, behavior, results, and relevance. By asking these 
questions after a period of time has passed, it will thus 


eliminate the potential positive bias that can result from 


Pe, 


good feelings in the course. In this way, NAC can find out 
the trainees’ perceptions of what really happens when they 
apply what they learn in the course to their jobs. 

2. Recommendation 2 

Recommendation Number 2 would require more effort but 
will be of more benefit to NAC. The suggestions listed below, 
if followed, should lead NAC to a more effective system for 
evaluating their training program. 

a. At a minimum, NAC should test the participants 
both before and after every course of instruction. This would 
help the training director in determining how much each of the 
participants have learned individually. Additionally, commen 
groups should be used whenever possible. This would help in 
determining whether the training was the cause of the increase 
or whether good mentoring or on-the-job experience was 


responsible for the increase. 


bx The next step, after step one is successfully 
implemented, is to measure behavioral changes. This step is 
more difficult. Changes in behavior do not take place over 


night and they do not occur simultaneously for all 
individuals. To measure changes in behavior, the trainees’ 
Supervisor, with input from the training director, must 
measure behavior before and after the program. Two things 
should be remembered: 1) allow enough time after the training 


for the change to occur, and 2) measure it several times to be 
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sure that the measurement is accurate. As in step one, use a 
control group whenever possible. Many factors can contribute 
to a change in behavior; raises or decreases in salary, 
attitude of the supervisor, or peer pressure from fellow 
employees. Withone a vcontrol “gGroup,= ties Crarming -airector 
would not be able to distinguish between changes due to the 
training or changes due to outside influences. 

These guidelines, which were developed by Donald 
Fenkpatrichk, were outlined 1m Chapter Lil of this study. They 
offer a reliable but often-times difficult means of evaluating 
training. Recommendation Number 2 offers the ideal type of 
evaluation and is the most difficult to implement. As a 
result, most organizations don’t do this much evaluation. 
However, Just because it may difficult does not mean that the 
evaluation should not be attempted. ins her currenc-srascad: 
climate it becomes increasingly important to do what you can 
to demonstrate the value of training. As noted by Jackson and 
Kulp, however, [Ref 18] getting a little bit of data is better 
than getting nothing at all. 

3. Recommendation 3 

The third recommendation is an attempt to determine 
whether it is worth NAC’s time, money, or effort to continue 
offering several of their courses. The training records 
indicate that seven of the twelve management training courses 


evaluated had fewer than eleven participants take the course 
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in the last year and a half. Is it worth NAC’s time and money 
to give a course to such a small portion of their scientists 
and engineers? The only way to know for sure is to conduct a 


cost benefit analysis. 


B. SUMMARY 

The main purpose of this study was to determine whether or 
not the management training program being offered by NAC to 
their scientists and engineers was perceived as beneficial. 
In order to find the answer to this question, four research 
questions were identified: 1) How relevant is management 
training to job requirements according to trainees; 2) How 
satisfied are trainees with specific management courses; 3) 
What is the perceived relationship between management training 
received and changes in job performance; and 4) What are the 
potential benefits of the management training for the NAC 
organization? A survey, with the above questions incorporated 
into it, was developed to help answer those questions. 

The results of the survey indicated that the trainees had 
positive responses to all of the above questions with the 
exception of question number one. This question was answered 
negatively 42 percent of the time for one course only: Basic 
Project Management. All of the other courses for this 
question were answered positively. However, due to the small 
number of surveys returned, the results for the entire study 


must be considered exploratory. 
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APPENDIX A 
NAVAL AVIONICS CENTER ORGANIZATIONAL CHART 
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APPENDIX B 


MANAGEMENT COURSES AVAILABLE TO NAC PERSONNEL 


The following is a list and short description of the 
courses available to NAC managers and management development 


candidates who would like suggestions for their IDP: 


J. AMA Self-Study Courses 


- General Management Skills 

- Marketing Management 

- Management and the Computer 

- Manufacturing Management 

- Managerial Finance and Control 


2. Supervisory Development (parts I and IT) 


Pai 
- Motivating the Average Employee 
- Improving Performances, Work Habits, and Attendance 
- Maintaining Improvement 
Swlakang Disciplinary Action 
- Handling Employee Complaints 


jes a oem 
- Establishing Goals and Objectives 
- Effective Delegation 
- Interviewing and Selection 
- Giving Bad News 
- Managing Change 
- Time Management 


3. Supervisory Institute (three day course) 


Daye 
- Leadership Styles 


fe lransition Co Supervision 
- Performance Standards/Appraisals 
- Enhancing Employee Motivation 


Day 2 
- Communications: Assertiveness, Praise, and Constructive 
Criticrem 
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- How to Delegate Effectively 
- Affixmative Action/EEO: Impact onmemper some 
- Nonverbal Aspects of Communication 


Day 3 
*- Managerial Aspects of Supervision 
- Setting Goals and Objectives 
- Personal Strategies for Working Smarter, Not Harder 


4. Advanced Management Workshop-Builds on concepts and skills 
introduced in both the "Supervisory Institute” andae7— 
"Supervisory Development" courses. Develops skills in: 


- Leading Group Process 

- Creative Problem Solving and Decision-Making 
- Negotiation 

- Interviewing 

- Conducting Effective Meetings 

- Managing Change 


5. IUPUI Public Management Institute Seminars 


- Making Effective Presentations 

- Teambuilding 

- Making Meetings More Productive 

- Meaningful Resolution of Conflict 
- The Power of Effective Listening 
- Advanced Management 


6. Evening classes at University of Indianapolis (most are 
AMA accredited) 


- How to Plan and Run Productive Meetings 

- Assertiveness Skills for Career/Personal Success 
- Getting Results with Time Management 

- How to Build Effective Work Teams 

- Personal Strategies for Managing Stress 


as AMA seminars 


- Improving Managerial Skills 

- Leadership Skills for New Managers 

- Time Management 

- Assertiveness Training 

Wists eainen, Speaking, and Listening fOr Successful 
Communication 

- And Approximately 90 more 
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D 


OPM Federal Executive Seminars and the Army’s Personnel 
Management for Executives course (two week seminars) 


Seminar for New Managers 

Management Development Seminar 
Managerial Competencies for Executives 
Executive Development Seminar 
Managing Money and Material Resources 


Indiana Business Seminars (IU School of Business-three day 


seminars) 


Cy 


Improving Management Skills 

Managing Writing for the Communication Age 
Production Management 

Sorat icti@al Process Control 


AMA’ s "Management Course" (IU). Taking three of the four 


units will satisfy much of the MDP requirement. 


nit des mOocue lng son. Rests: Key Management Actions 
Organizational Growth 


Strategic Planning 
Team Planning Process 
Creativity and Teamwork 


Unit 2, Using Analytical Tools: Financial Planning and 


Management 


PrOoritabi lity 

Managing the Profit Center 
Measures of Performance 
Break-Even Point Analysis 


Unit 3, Leading Your People: Manager’s Role as Leader and 


Motivator 


Bxecutive Leadership 
Delegation 

Negotiation Skills 
Selection and Interviewing 
Performance Appraisal 


Cho 5 ome Strategic Marketing Management: Competing 


Successfully and Satisfying Customers 


Strategic Thinking 

Dealing with Competition 

Customer Satisfaction and Quality 
Quality Management 


67 


This inventory of available courses is not an all inclusive 
list, and employees are not limited to just these courses. 
Since every worker has different educational ideas and goals, 
employees are encouraged to work closely with their 
supervisors in order to develop a plan that best fits their 


Own needs. 
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APPENDIX C 


MANAGEMENT TRAINING QUESTIONNAIRE 


es, 


NAC SCIENTIST AND ENGINEER TRAINING SURVEY 


Dear Survey FParttetpane: 


Thank you for your willingness to participate in the survey 
of Scientist and Engineer Communities at the Naval Avionics 
Center (NAC). This survey is part of an ongoing study sponsored 
by NAC’s Personnel Department to improve its customer service. 
Participation such as yours will assure that all viewpoints are 
reflected in the conclusions and recommendations resulting from 
the study. 


The main purpose of the study is to obtain individual 
perceptions of management training from those receiving the 


training, i-.e., scientists and engineers. It is an opportune. 
to register your observations, concerns, and satisfactions on a 
number of training-related topics and issues. This survey will 


allow us to see how the scientist and engineer communities feel 
about these issues. 


The following questionnaire was custom designed for NAC. A few 
questions cover standard demographic characteristics questions 
which help us with the statistical analysis of the data. eae 
most of the items reflect issues of specific concern to NAC as 
identified through interviews. 


All responses will be held in complete confidentiality, so 


please be frank and honest in your answers. The questionnaires 
will be administered and collected by a member of the Naval 
Postgraduate School. Your answers will be combined with others 


so that no individual responses will be reported or made 
available to anyone. 


The survey should take about 30 minutes to complete. Once 
the data are analyzed, a report of findings will be prepared and 
made available to all interested personnel through Department 
SNOOP 

Thank you for your cooperation. 
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Naval Postgraduate School Naval Postgraduate School 


GENERAL SURVEY INSTRUCTIONS 


The following information is needed to help us with the 
statistical analysis of the data. All of your responses are 
STRICTLY CONFIDENTIAL; no individual responses will be reported. 
Data will be agqgregated and average ratings calculated. This 
information will allow comparisons to be made among different 
groups of employees. 


In responding to the questions in this survey, we ask that 
you consider only NAC-sponsored on-site or off-site management 


training courses. We do not want responses which make reference 
to mandatory-type administrative classes which everyone is 
required to attend (1.e., safety, CPR, sexual harassment, etc.). 


Most of the questions in this survey will ask about your 
perceptions of the training you’ve received. These will be open 
questions which allow you to formulate your own answers. 


The survey is broken down into 2 sections: 


SECTION EXPECTED COMPLETION TIME 
SECTION . 1 -— DEMOGRAPHICS Ss = 2) ZMENUTES 
SECTION II - MANAGEMENT TRAINING TS = -20CMINUTES 


The entire survey should take between 20 and 30 minutes, 
depending upon the length of your responses. 


Again, thank you for your cooperation. 


SECTION I - DEMOGRAPHICS 


PLEASE CIRCLE THE LETTER BESIDE THE APPROPRIATE RESPONSE FOR 
FACH QUESTION OR WRITE IN THE CORRECT INFORMATION. 


1. Are you: a. Male b. female 
2. How many years have you worked at NAC? 
a» under 1 oe ae, C..2 6 = ad 
Oi, ld Sales aS. ena 76 £. over 20 
3. Your paygrade is? GS/GM - 


4. What is your job series number (e.g., 0855 for Safety 
Engineer) ? 


5. What department/division/branch do you work for? 
Dept : Daly: Br: 
6. What is your supervisory experience? 
a. I am not a formally designated supervisor and I have not 


supervised other persons during the past year. 


b. I am not a formally designated supervisor but I have 
supervised other persons during the past year. 


c. I have been a formally designated supervisor for less 
je loiehou lbaealdevehe - 


d. I have been a formally designated supervisor for 1 to 5 
years. 


iD 


I have been a formally designated supervisor for 5 to 
Cen years. 


f. I have been a formally designated supervisor for more 
than 10 years. 


SECTION TL - MANAGEMENT TRAINING 


PLEASE CIRCLE THE LETTER BESIDE THE APPROPRIATE 
RESPONSE FOR EACH QUESTION OR WRITE IN THE CORRECT INFORMATION. 
BE AS SPECIFIC AS POSSIBLE. 


Paecourse Title. 
2. About how many months has it been since you have had this 
teaming ? 

a. under ] 6 ee ne: Cc. G4 4-26 

Gam 1- =o So 1012 EE. overgZ 


3. Why did you take this course? 


4. Describe any non-observable ways you have changed as a result 
CEetnis training, for example, in your thinking, level of 
understanding, attitude, etc. 


MANAGEMENT TRAINING 


5. Describe any behavioral ways you have changed as a result of 
this training: what do you do differently now on your jcb acme 
result of having taken this course? 


6. Please consider your answers to questions 4 and 5 and 
describe how you think these training outcomes have benefitted 
NAC. 


7. Have you had any difficulty in applying anything you learned 
in the course to your job? 


a. No 


b. Yes (If yes, why?) 


MANAGEMENT TRAINING 


READ EACH STATEMENT CAREFULLY AND THEN CIRCLE YOUR ANSWER 
ACCORDING TO THE FOLLOWING SCALE: 


1--Strongly Disagree--I strongly disagree with the statement 
2--Disagree--I disagree with the statement, but not so strongly 
3--Neutxral--I am neutral toward the statement 

A--Agree--I agree with the statement, but not so strongly 
5--Stxrongly Agree--I strongly agree with the statement 


8. How do you feel about the following statements? 
a. This course was relevant to my job, 
1.¢., it addressed problems and 
Pisces JT Nenceunter 1O-My wjOlt so == saa ea I 2 32 2 
b. This course met the objectives of 
improving my abilities in this area.------- ie ee en 


c. NAC has benefitted from the management 
Catt Arn te oaced od == S34 SSS = ae See oe ee 


d. Overall, I was satisfied with this 
COU LS ete = SS ee Se Se Se aS eee vor oo. 64 


PLEASE CONSIDER ONE LAST QUESTION: 


oO" Should NAC continue to offer this course? 
a. yes 


men AIO) (ee no, e why not?) 
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